Home | | Latest | About | Random
# 08 "Review" of basic matrix terminologies and operations - addition, multiplication, scaling and transpose.
I'd like to "review" first how some matrix operations works, and some of its basic properties. I use the word "review" as it is part of [California K-12 Math Curriculum for Pre-Calculus (see pages 8, 9)](https://www.cde.ca.gov/ci/ma/cf/documents/mathfwprecalculus.pdf). But perhaps it has been a distant past. So let us briefly review it.
## Size of a matrix.
First a matrix consists a table of numbers, consists of **rows** (each row has numbers that go across horizontally), and **columns** (each column has numbers that go down vertically). And to describe a matrix, we say it of size $n\times k$ if it has $n$ rows and $k$ columns: $$
\text{$n$ rows}\left\{\begin{matrix}
\ \\ \\
\ \\
\ \\
\end{matrix} \right.\ \underbrace{\begin{bmatrix}
a_{11} & a_{12} & \cdots & a_{1k} \\
a_{21} & a_{22} & \cdots & a_{2k} \\
\vdots \\
a_{n1} & a_{n2} & \cdots & a_{nk}
\end{bmatrix}}_{\text{$k$ columns}}
$$In other words, an $n \times k$ matrix is $n$-tall and $k$-fat. And you always describe a matrix by how tall it is first, then how fat -- just to be polite.
So for instance $\begin{bmatrix}1 & 2 & 3\\4 & 5 & 6\end{bmatrix}$ is a $2 \times 3$ matrix; while $\begin{bmatrix}1 & 2 \\3 & 4 \\ 5 & 6\end{bmatrix}$ is a $3\times 2$ matrix. This is the size naming convention we adopt.
Sometimes we will use the phrase "column vector" to describe a matrix that is a single column, so a matrix of size $n\times 1$; and the word "row vector" to describe a matrix that is a single row, so a matrix of size $1\times k$. However, later the word "vector" will be more general so let us just keep in mind this additional jargon.
By the way, yes we have $1\times1$ matrices, like $\begin{bmatrix}3\end{bmatrix}$. Sometimes, and very carefully sometimes, we can treat it like a scalar, but this is context dependent.
And to be even more technical -- are there $0\times 0$ or $0 \times k$ or $n\times 0$ matrices. Yes... but we have to be careful what they mean and how to use them. Perhaps we will encounter them later. Note, a $0\times0$ matrix is not the same as a $0\times 4$ matrix or a $0\times 5$ matrix, they are all different. But what exactly are these things? To really make sense of these we need the idea of "vectorspaces" and "linear transformations" later.
Can the size $n$ or $k$ be negative? No. They need to be non-negative integers. And most typically, they are just both positive integers.
## Shape of the brackets around a matrix?
Sometimes you may see a matrix written like the following two ways: $$
\begin{bmatrix}
1 & 2 & 3 \\
4 & 5 & 6
\end{bmatrix}
\quad \text{or}\quad
\begin{pmatrix}
1 & 2 & 3 \\
4 & 5 & 6
\end{pmatrix}
$$Both are these are acceptable and generally do not cause any confusion. So you can use them interchangeably. However, avoid using curly braces $\{\cdots \}$ or vertical bars $|\cdots|$ to enclose your matrices. Sometimes they have special meaning and people might get confused. Using square brackets or rounded parenthesis people will at least default to "ok you are maybe describing a matrix here". This is our convention.
## Entries in a matrix.
For a matrix $A$, we sometimes write $A_{ij}$ or $A_{i,j}$ or $A[i,j]$ or $(A)_{ij}$ or $[A]_{ij}$ to denote the $i$-th row, $j$-th column entry of the matrix $A$. Or if $i$ and $j$ are double digits or more then we write $A_{i,j}$ like that to avoid confusion. Or if we write $A=[a_{ij}]$, then this is suggesting that we use the symbol $a_{ij}$ to denote the $i$-th row, $j$-th column entry. Together this is called the $(i,j)$-th entry of the matrix.
For example, suppose matrix $A=\begin{bmatrix}3 & 2 & 4\\ 0 & 9 & 1\end{bmatrix}$, then $A_{12}=2$ and $A_{21}=0$. The entry $A_{13} = 4$ and the entry $A_{31}$ does not exist.
Or if we say, we consider some matrix $A=[a_{ij}] = \begin{bmatrix}3 & 2 & 4\\ 0 & 9 & 1\end{bmatrix}$, then this is implicitly suggesting us to use the symbol $a_{ij}$ to describe the entries. So here $a_{12}=2$ and $a_{21}=0$, etc.
These are just some common ways. But watch out for variations, and try to be context aware.
## Adding matrices together.
If we have two matrices $A,B$ and they both have the same size, say both $n\times k$, then we can add them together **entry by entry**. So the $i,j$-th entry of $A+B$ is the $i,j$-th entry of $A$ add to $i,j$-th entry of $B$.
If $A$ and $B$ have different sizes, then we cannot add them together and their sum is not defined.
For example, $$
\begin{bmatrix}
4 & 3 & 2 \\
3 & 4 & 1
\end{bmatrix} + \begin{bmatrix}
-2 & 3 & 5 \\
1 & 0 & 1
\end{bmatrix} = \begin{bmatrix}
2 & 6 & 7 \\
4 & 4 & 2
\end{bmatrix}.
$$
And again, if the sizes do not match, we cannot add the matrices together. So, $$
\begin{bmatrix}
1 \\
2 \\
3
\end{bmatrix} + \begin{bmatrix}
1 & 3 \\
3 & 2 \\
2 & 0
\end{bmatrix}\quad\text{is undefined}
$$and so is $$
\begin{bmatrix}
3 & 3 & 4 \\
2 & 2 & 1
\end{bmatrix} + \begin{bmatrix}
5 & 3 \\
2 & 2 \\
2 & 1
\end{bmatrix}\quad \text{also undefined.}
$$
## Multiply matrices together. This is important.
If we have two matrices $A$ and $B$ together, then we can multiply them together if their "inner sizes match up". That is, if $A$ is of size $n\times k$ and $B$ is of size $m\times p$, then the product $AB$ is defined if and only if $k=m$:
```tikz
\begin{document}
\begin{tikzpicture}
\node at (0,0) {\(\mathop{A}\limits_{n\times k} \mathop{B}\limits_{m\times p}\)};
\draw (-0.15,-0.3) -- (-0.15,-0.8) -- (0.13,-0.8) -- (0.13,-0.3);
\node at (0,-1) {\(k\) and \(m\) need to be the same to multiply};
\end{tikzpicture}
\end{document}
```
That is, the number of columns of $A$ needs to be the same as the number of rows of $B$ for their product $AB$ to exist and be defined.
For example, if $A$ is $3\times 5$ and $B$ is $5\times 4$, then the matrix product $AB$ is defined, however the product $BA$ is not defined! So matrix multiplication is **not commutative** in general.
Now, if we have $A$ is of size $n\times k$ and $B$ is of size $k\times p$, the resulting matrix product $AB$ is of size $n\times p$ (the other sizes). The entries of $AB$ is obtained by taking the dot product of a row of $A$ with a column of $B$ (and since the sizes match up correctly, this is doable). In particular, the $i,j$-th entry of $AB$ is the dot product of the $i$-th row of $A$ with the $j$-th column of $B$.
To illustrate an example: Let us multiply $A=\begin{bmatrix}3 & 2 & -1 \\5 & -2 & 7\end{bmatrix}_{2\times3}$ to the left of $B=\begin{bmatrix}1 & 4 & 2 & 2\\2 & 0 & 3 & 1\\5 & 2 & 1 & -2\end{bmatrix}_{3\times 4}$.
By the way, $AB$ is defined here (which is going to be $2\times 4$), but $BA$ is **not defined**.
Ok let us look at it step-by-step of multiplying $AB$: First we have $$
\begin{bmatrix}\colorbox{lightblue}3 & \colorbox{lightblue}2 & \colorbox{lightblue}{\(-1\)} \\5 & -2 & 7\end{bmatrix}\begin{bmatrix}\colorbox{lightblue}1 & 4 & 2 & 2\\\colorbox{lightblue}2 & 0 & 3 & 1\\\colorbox{lightblue}5 & 2 & 1 & -2\end{bmatrix}=\begin{bmatrix}
\colorbox{lightblue}{\(3(1)+2(2)-1(5)\)} & \ast & \ast & \ast\\
\ast & \ast & \ast & \ast
\end{bmatrix}=\begin{bmatrix}
\colorbox{lightblue}2 & \ast & \ast & \ast\\
\ast & \ast & \ast & \ast
\end{bmatrix}
$$
which we get the $1,1$ entry of $AB$, using the $1$st row of $A$ and $1$st column of $B$. Next,$$
\begin{bmatrix}\colorbox{pink}3 & \colorbox{pink}2 & \colorbox{pink}{\(-1\)} \\5 & -2 & 7\end{bmatrix}\begin{bmatrix}1 & \colorbox{pink}4 & 2 & 2\\2 & \colorbox{pink}0 & 3 & 1\\5 & \colorbox{pink}2 & 1 & -2\end{bmatrix}=\begin{bmatrix}
2 & \colorbox{pink}{\(3(4)+2(0)-1(2)\)}& \ast & \ast\\
\ast & \ast & \ast & \ast
\end{bmatrix}=\begin{bmatrix}
2 & \colorbox{pink}{10} & \ast & \ast\\
\ast & \ast & \ast & \ast
\end{bmatrix}
$$which computes the $1,2$ entry of $AB$. Next, we have $$
\begin{bmatrix}\colorbox{lightgreen}3 & \colorbox{lightgreen}2 & \colorbox{lightgreen}{\(-1\)} \\5 & -2 & 7\end{bmatrix}\begin{bmatrix}1 & 4 & \colorbox{lightgreen}2 & 2\\2 & 0 & \colorbox{lightgreen}3 & 1\\5 & 2 & \colorbox{lightgreen}1 & -2\end{bmatrix}=\begin{bmatrix}
2 & 10 & \colorbox{lightgreen}{\(3(2)+2(3)-1(1)\)} & \ast\\
\ast & \ast & \ast & \ast
\end{bmatrix}=\begin{bmatrix}
2 & 10 & \colorbox{lightgreen}{11} & \ast\\
\ast & \ast & \ast & \ast
\end{bmatrix}
$$And we finish showing the rest: $$
\begin{bmatrix}\colorbox{yellow}3 & \colorbox{yellow}2 & \colorbox{yellow}{\(-1\)} \\5 & -2 & 7\end{bmatrix}\begin{bmatrix}1 & 4 & 2 & \colorbox{yellow}2\\2 & 0 & 3 & \colorbox{yellow}1\\5 & 2 & 1 & \colorbox{yellow}{\(-2\)}\end{bmatrix}=\begin{bmatrix}
2 & 10 & 11 & \colorbox{yellow}{10}\\
\ast & \ast & \ast & \ast
\end{bmatrix}
$$ $$
\begin{bmatrix}3 & 2 &-1 \\ \colorbox{gold}5 & \colorbox{gold}{\(-2\)} & \colorbox{gold} 7\end{bmatrix}\begin{bmatrix}\colorbox{gold}1 & 4 & 2 & 2\\\colorbox{gold}2 & 0 & 3 & 1\\\colorbox{gold}5 & 2 & 1 & -2\end{bmatrix}=\begin{bmatrix}
2 & 10 & 11 & 10\\
\colorbox{gold}{36} & \ast & \ast & \ast
\end{bmatrix}
$$ $$
\begin{bmatrix}3 & 2 &-1 \\ \colorbox{tan}5 & \colorbox{tan}{\(-2\)} & \colorbox{tan} 7\end{bmatrix}\begin{bmatrix} 1 & \colorbox{tan} 4 & 2 & 2\\ 2 & \colorbox{tan}0 & 3 & 1\\5 & \colorbox{tan} 2 & 1 & -2\end{bmatrix}=\begin{bmatrix}
2 & 10 & 11 & 10\\
36 & \colorbox{tan}{34} & \ast & \ast
\end{bmatrix}
$$ $$
\begin{bmatrix}3 & 2 &-1 \\ \colorbox{cyan}5 & \colorbox{cyan}{\(-2\)} & \colorbox{cyan} 7\end{bmatrix}\begin{bmatrix} 1 & 4 & \colorbox{cyan}2 & 2\\ 2 & 0 & \colorbox{cyan} 3 & 1\\5 & 2 & \colorbox{cyan}1 & -2\end{bmatrix}=\begin{bmatrix}
2 & 10 & 11 & 10\\
36 & 34 & \colorbox{cyan}{11}& \ast
\end{bmatrix}
$$ $$
\begin{bmatrix}3 & 2 &-1 \\ \colorbox{turquoise}5 & \colorbox{turquoise}{\(-2\)} & \colorbox{turquoise} 7\end{bmatrix}\begin{bmatrix} 1 & 4 & 2 & \colorbox{turquoise} 2\\ 2 & 0 & 3 &\colorbox{turquoise} 1\\5 & 2 & 1 & \colorbox{turquoise} {\(-2\)}\end{bmatrix}=\begin{bmatrix}
2 & 10 & 11 & 10\\
36 & 34 & 11& \colorbox{turquoise}{\(-6\)}
\end{bmatrix}
$$It is good to practice how to multiply matrices together. We will do this quite a bit. And we should note, there are other ways of viewing this multiplication, to which we will do later.
**Something to think about.** Can you come up with two matrices $A$ and $B$, where $AB$ is defined, and $BA$ is also define. But $AB \neq BA$?
## Power of a matrix.
If a matrix $A$ is a **square matrix**, namely it is of size $n\times n$, where there is equal number of rows as columns, then the product $AA$ is defined, as well as $(AA)A$. For repeated matrix product with itself, we use the power notation, so $$
A^{p}=\underbrace{AA\cdots A}_{\text{\(p\) times}}.
$$ The idea of matrix power will be useful for us later. So keep this in mind.
## Scaling of a matrix.
Given a matrix $A$ and a scalar $c$, we can compute the matrix scaling $cA$, where we just scale each entry of $A$ by $c$. For example, $$
3\begin{bmatrix}
1 & 4 & 2 \\
3 & -5 & -3 \\
0 & 7 & -4
\end{bmatrix}=\begin{bmatrix}
3 & 12 & 6 \\
9 & -15 & -9 \\
0 & 21 & -12
\end{bmatrix}
$$
## Transpose of a matrix.
The **transpose** of a matrix $A$ of size $n\times k$ is a new matrix called $A^{T}$ which is of size $k\times n$, where the $i$-th row of $A$ becomes the $i$-th column of $A^{T}$.
So if $$
A= \begin{bmatrix}
1 & 4 & 2 \\
-2 & 0 & 5
\end{bmatrix}, \quad \text{then } A^{T}=\begin{bmatrix}
1 & -2 \\
4 & 0 \\
2 & 5
\end{bmatrix}.
$$Here $A$ is $2\times 3$, and $A^{T}$ is $3\times 2$.
Sometimes, we denote the transpose of $A$ as $A^{T}$ or $A^{t}$, depends on style and preference. Notice, for any matrix, its transpose is always defined -- we are just taking the rows of one, and turning them into columns (the order, of course, matters). We will see how transposes will be useful later.